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(54) DEVICE AND METHOD FOR PROCESSING INFORMATION 

(57)Abstract: 

PROBLEM TO BE SOLVED: To incorporate information 
for deciding the order of reproduction of attached 
information into electronic mail information and to easily 
reproduce information in the desired order of 
reproduction. 

SOLUTION: An attached file 404 including image data is 
embedded in electronic mail data including a header part 
401 and text data 402. Header information 403 is added 
to each attached file, and the preparation date of the 
corresponding attached file is added to the header 
information 403 as information indicating the mutual 
relation of the attached files in the electronic mail data. 
In a device which receives the electronic mail data, at 
the time of reproducing an image based on the attached 
file, the order of reproduction is decided based on the 
preparation date included in the header information, and 
the display in a slide format is operated in the decided 
order of reproduction. 
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* NOTICES * 

Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer.So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1] The information processor characterized by having the file appending means which 
embeds an attached file into electronic mail data, the correlation means which embeds the 
correlation information showing the mutual relation of the attached file appended with the 
aforementioned file appending means into the aforementioned electronic mail data, and a 
transmitting means to transmit the electronic mail data obtained with the aforementioned 
correlation means. 

[Claim 2] The aforementioned attached file is an information processor according to claim 1 
characterized by being the file of image information. 

[Claim 3] The aforementioned attached file is an information processor according to claim 1 
characterized by being the file of speech information. 

[Claim 4] The aforementioned correlation information is an information processor according to 
claim 1 characterized by being a time entry in connection with the aforementioned attached file. 
[Claim 5] The aforementioned correlation information is an information processor according to 
claim 4 characterized by being the date and time of creation of the aforementioned attached file. 

[Claim 6] The aforementioned correlation means is an information processor according to claim 1 
characterized by extracting the time stump from the attribute information on the aforementioned 
attached file, and embedding by making the this extracted time stump into the aforementioned 
correlation information at the aforementioned electronic mail data. 

[Claim 7] The aforementioned correlation means is an information processor according to claim 1 
characterized by relating based on manual operation, generating information, and appending to 
the aforementioned electronic mail data. 

[Claim 8] The information processor characterized by providing the following. A judgment means 
to judge whether the attached file is embedded at the received electronic mail data. An 
extraction means to extract the correlation information showing the mutual relation of the 
aforementioned attached file from the electronic mail data concerned. A determination means to 
determine the sequence which was extracted with the aforementioned extraction means and 
which relates and reproduces the data of the aforementioned attached file based on information. 
The reproduction means which carries out the reproduction output of the data of the 
aforementioned attached file in the sequence determined with the aforementioned determination 
means. 

[Claim 9] The aforementioned correlation information is an information processor according to 
claim 8 characterized by being the time entry stored in each of the aforementioned attached file 
by matching. 

[Claim 10] It is the information processor according to claim 8 which the aforementioned 
attached file is an image data file, and is characterized by for the aforementioned reproduction 
means choosing the aforementioned attached file based on the sequence determined with the 
aforementioned determination means, and carrying out the reproduction output of the image one 
by one. 

[Claim 1 1] It is the information processor according to claim 8 characterized by for the 
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aforementioned attached file being an image data file, and for the aforementioned reproduction 
means arranging the image based on the data of the aforementioned attached file based on the 
sequence determined with the aforementioned determination means, and carrying out a 
reproduction output. 

[Claim 12] The information processing method characterized by having the file appending 
process which embeds an attached file into electronic mail data, the correlation process which 
embeds the correlation information showing the mutual relation of the attached file appended at 
the aforementioned file appending process into the aforementioned electronic mail data, and the 
transmitting process which transmits the electronic mail data obtained at the aforementioned 
correlation process. 

[Claim 13] The aforementioned attached file is the information processing method according to 
claim 12 characterized by being the file of image information. 

[Claim 14] The aforementioned attached file is the information processing method according to 
claim 12 characterized by being the file of speech information. 

[Claim 15] The aforementioned correlation information is the information processing method 
according to claim 12 characterized by being a time entry in connection with the aforementioned 
attached file. 

[Claim 16] The aforementioned correlation information is the information processing method 
according to claim 15 characterized by being the date and time of creation of the 
aforementioned attached file. 

[Claim 17] The aforementioned correlation process is the information processing method 
according to claim 12 characterized by extracting the time stump from the attribute information 
on the aforementioned attached file, and embedding by making the this extracted time stump 
into the aforementioned correlation information at the aforementioned electronic mail data. 
[Claim 18] The aforementioned correlation process is the information processing method 
according to claim 12 characterized by relating based on manual operation, generating 
information, and appending to the aforementioned electronic mail data. 

[Claim 19] The information processing method characterized by providing the following. The 
judgment process which judges whether the attached file is embedded at the received electronic 
mail data. The extraction process which extracts the correlation information showing the mutual 
relation of the aforementioned attached file from the electronic mail data concerned. The 
determination process which determines the sequence which was extracted at the 
aforementioned extraction process, and which relates and reproduces the data of the 
aforementioned attached file based on information. The reproduction process which carries out 
the reproduction output of the data of the aforementioned attached file in the sequence 
determined at the aforementioned determination process. 

[Claim 20] The aforementioned correlation information is the information processing method 
according to claim 19 characterized by being the time entry stored in each of the 
aforementioned attached file by matching. 

[Claim 21] It is the information processing method according to claim 19 which the 
aforementioned attached file is an image data file, and is characterized by for the 
aforementioned reproduction process choosing the aforementioned attached file based on the 
sequence determined at the aforementioned determination process, and carrying out the 
reproduction output of the image one by one. 

[Claim 22] It is the information processing method according to claim 19 characterized by for the 
aforementioned attached file being an image data file, and for the aforementioned reproduction 
process arranging the image based on the data of the aforementioned attached file based on the 
sequence determined at the aforementioned determination process, and carrying out a 
reproduction output. 

[Claim 23] The storage which carries out [ having the code of the correlation process which 
embeds the correlation information which expresses the mutual relation of the attached file 
appended at the aforementioned file appending process as the code of the file appending 
process of storing the control program for making a computer generating electronic mail data 
and that are a storage and this control program embeds an attached file into electronic mail data 
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into the aforementioned electronic mail data, and the code of the transmitting process which 
transmits the electronic mail data obtained at the aforementioned correlation process, and J as 

the feature. . . 

[Claim 24] The storage which is characterized by providing the following and which stores the 
control program for indicating the electronic mail data by visible in a computer. The judgment 
process code which judges whether the attached file is embedded at the electronic mail data 
which this control program received. The code of the extraction process which extracts the 
correlation information showing the mutual relation of the aforementioned attached file from the 
electronic mail data concerned. The code of the determination process which determines the 
sequence which was extracted at the aforementioned extraction process, and which relates and 
reproduces the data of the aforementioned attached file based on information. The code of the 
reproduction process which carries out the reproduction output of the data of the 
aforementioned attached file in the sequence determined at the aforementioned determination 
process. 



[Translation done.] 
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* NOTICES * 

Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 

DETAILED DESCRIPTION 



[Detailed Description of the Invention] 

[0001] . . 

[Field of the Invention] This inventions are the Internet, personal computer communications, etc., 
and relate to the information processor which can transmit and receive an electronic mail server 
and an E-mail, and its method. 

[0002] . , c .. 

[Description of the Prior Art] The equipment which generally transmits and receives an E-mail 
transmits and receives the E-mail information which consists of a header unit specified by 
RFC822 (Request For Comment 822) etc., and a body part using communications protocols, such 
as SMTP (Simple Mail Transfer Protocol) and POP3 (Post Office Protocol Version 3). 
[0003] Moreover, when carrying out two or more storing (henceforth appending) of text 
information or the image information to an E-mail, generally the technique of carrying out block 
division is used using Multipart description of the MIME (Multipurpose Internet Mail Extensions) 
format specified to RFC2045 or RFC2046. 

[0004] Under an above-mentioned agreement, when transmitting and receiving an E mail, 
determining the classification (a still picture or animation) of the image information to append is 
left to a transmitting side. 

[0005] ,. . , 

[Problem(s) to be Solved by the Invention] However, correlation between appending information 
cannot be performed in the E-mail information transmitted and received. For example, when 
appending information is image information, the order of reproduction cannot be specified. For 
this reason, the receptacle side of an E-mail only displays the information appended with the 
mere appearance order of appending information. 

[0006] For example, since correlation between image information was not able to be performed 
as mentioned above when two or more still pictures which carried out the seriography 
(henceforth continuous shooting) are appended to an E-mail and indicate by slide like a paper 
play by the receiving side with a digital still camera etc., the image information which processed^ 
image information into slide form and processed it beforehand needed to be appended to the E- 
mail. Or when it is going to output they-appended image information in index form (a display, 
printing), what specifies the display order cannot be performed. 

[0007] Although a means to substitute animation information was also considered in order to 
specify the reproduction sequence of a still picture image, when treated by the animation, 
amount of information increased, and there was a problem of raising the load of a network 
circuit. Moreover, in order to have processed and reproduced animation information, there was 
also a problem of needing the memory resources of the software of exclusive use or many. 
[0008] this invention is made in view of the above-mentioned problem, makes it possible to 
include the information for determining the reproduction sequence of appending information in E- 
mail information, and aims at enabling it to perform information reproduction in desired 
reproduction sequence easily. 
[0009] 

[Means for Solving the Problem] The information processor by one mode of this invention for 
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solving the above-mentioned technical problem is equipped with the following composition. That 
is, it has the file appending means which embeds an attached file into electronic mail data, the 
correlation means which embeds the correlation information showing the mutual relation of the 
attached file appended with the aforementioned file appending means into the aforementioned 
electronic mail data, and a transmitting means to transmit the electronic mail data obtained with 
the aforementioned correlation means. 

[0010] Moreover, the information processor by other modes of this invention for solving the 
above-mentioned technical problem is equipped with the following composition. That is, it has a 
judgment means judge whether an attached file is embedded at the electronic mail data which 
received, an extraction means extract the correlation information showing the mutual relation of 
the aforementioned attached file from the electronic mail data concerned, a determination means 
determine the sequence which was extracted with the aforementioned extraction means and 
which relates and reproduces the data of the aforementioned attached file based on information, 
and a reproduction means carry out a reproduction output in the data of the aforementioned 
attached file at the sequence determined with the aforementioned determination means. 
[0011] 

[Embodiments of the Invention] Hereafter, the suitable operation gestalt of this invention is 
explained with reference to an attached drawing. 

[0012] Drawing 1 is the block diagram showing the composition of the information processor 
which functions as an E-mail receiving set by this operation gestalt. In drawing 1 , 1 1 is CPU, 
performs the control program stored in ROM12 or RAMI 3, and realizes various processings. 12 is 
ROM and stores the control program and the various data which are performed by CPU1 1. 13 is 
RAM and functions as main memory of CPU1 1. 14 is external storage and stores the control 
program for realizing E-mail transceiver control of this operation gestalt later mentioned with 
reference to a flow chart, an image data, etc. As external storage 1 4, a floppy (registered 
trademark) disk, a hard disk, CD-ROM, etc. are mentioned. In addition, the control program 
stored in external storage 14 is loaded to RAM 13 if needed, and is performed by CPU1 1. 
[0013] 15 is an input unit and possesses a keyboard, a mouse, etc. 16 is a display and performs 
various displays under control of CPU1 1. 17 is a printer. Moreover, 18 is an interface for making 
connection with the Internet. For example, a modem will be used when connecting with the 
Internet via the telephone line. 

[0014] Drawing 2 is a block diagram explaining the functional composition of the E-mail 
transceiver processing by this operation gestalt. In drawing 2 , 1 is the E-mail transmitting 
section which transmits E-mail information to an electronic mail server, and SMTP is used for it 
as a communications protocol with this operation gestalt (with this operation gestalt, as shown in 
drawin g 1 , an E-mail is transmitted through an interface 18 and the Internet). In addition, about 
the detail of this communications protocol, since it is well-known at RFC, detailed explanation is 
omitted. 

[0015] 2 is the E-mail processing section which creates and interprets the E-mail information 
which consists of a header and the body. Since the detail about the composition of an E-mail is 
well-known in RFC822 grade, detailed explanation is omitted. The multi-part processing section 
which interprets the information by which carried out block division, and stored two or more text 
information and image information in the E-mail body, or block division was carried out, and 24 
are [ 22 / the E-mail header processing section in which 21 performs creation and an 
interpretation of the header unit of an E-mail, and ] the correlation processing section which 
carries out with / of two or more image information / relation in the E-mail body processing 
section which performs creation and an interpretation of the E-mail body, and 23. With this 
operation form, it is considering as a form using the date and time of creation of an attached file 
as a means which associates image information. 

[0016] 3 is a mail receive section which pulls out E-mail information from an electronic mail 
server. With this operation form, POP3 was used as a communications protocol (with this 
operation form, as shown in drawing 1 , an E-mail is received through an interface 18 and the 
Internet). About detailed explanation of a protocol, since it is well-known at RFC, it omits. 
[0017] The E-mail display by which 4 outputs E-mail information to a screen, paper, etc., and 41 
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are the image output sections which display and print associated image information in a display 
or index form in slide form. 

[0018] Operation of the E-mail transmitter-receiver of this operation form equipped with the 
above composition is explained. 

[0019] Drawin g_3 is a flow chart explaining the transmitting procedure of the electronic mail data 
based on this operation form. First, in Step S30, it investigates whether the transmitting start of 
an E-mail was directed. When a transmitting start is directed, it progresses to Step S31, and the 
E-mail information specified by the E-mail transmitting section 1 is transmitted. 
[0020] On the other hand, if the transmitting start is not directed, it progresses to Step S32 and 
user operation investigates whether it is file appending operation. If it is operations other than 
file appending operation, it will progress to Step S33 and processing corresponding to the 
contents of operation will be performed. Here, according to alter operation, such as the 
receiver's address, the E-mail header processing section 21 generates the header unit of 
electronic mail data, or the E-mail body processing section 22 generates this portion of an E- 
mail according to the alter operation of the E-mail text, for example. 

[0021] When directions of file appending are performed at Step S32, appending of the file 
specified by the user is performed by the multi-part processing section 23. First in Step S34 
the attached file related attachment processing section 24 performs correlation between 
attached files. With this operation form, since the attached file was associated by the date and 
time of creation, it considered as the method which embeds and associates "X-content-Create- 
DateiFri, 11 Dec 199817:27:41+0900", etc. as header information of appending information In 
addition, the date and time of creation of an attached file may be extracted from the property of 
the file specified that it should append, and may be generated and added automatically At Step 
S35, the multi-part processing section 23 embeds the specified image information or the text 
information into the electronic mail data concerned as an attached file. In addition although it is 
embedding into electronic mail data immediately with this operation form after associating the 
specified information, it does not restrict to this, it relates and header information is created and 
when transmitting d.rections are performed, you may make it composition which embeds the' 
specified information into electronic mail data. 

[0022] Drawing 4 is drawing showing the example of data composition of the electronic mail data 
based on this operation form. In c__av__n__4 . 401 is the header unit of the electronic mail data 
concerned, is based on an operation input from a user, and is generated and added by the E-mail 
header processing section 21. 402 is these data of the electronic mail data concerned, is based 
on an operation input from a user, and is generated and added by the E-mail body processing 
section 

[0023] 403 is the header information of the attached file generated at the above-mentioned step 
S34, and the type (an image, text, etc.) of the file concerned, the information on the date and 
time ^ creation etc. are stored. 404 is an attached file main part. Header information 403 and an 
attached file 404 are embedded by the multi-part processing section 23 at electronic mail data 
as otep S35 shows to drawing 4 . 

[0024] D^awing^ is a flow chart explaining the procedure of the reception mail by this operation 
gestalt. In drawing 5 , Step S40 asks the existence of reception mail to an electronic mail server 
If there .s reception mail, it will progress to Step S41. At Step S41, an E-mail is received through 
the E-mail receive section 3, the header information of the reception mail concerned is analyzed 
after that, and it progresses to Step S42. 

[0025] At Step S42, as a result of analyzing header information, it judges whether an attached 
file exists in the E-mail which received. When an attached file does not exist, it progresses to 

rnn?fi? n Tu T ? , disP ' ay Drocessin & Printing processing, etc. are performed. 

LUUZ6J On the other hand, when appending information exists at Step S42. it progresses to Step 
b44, and restoration processing of appending information is performed At Step S45 it 
investigates that it was finished whether processing all appending information (block : appending 
information header + appending information). If there is other appending information, Step S44 
will be processed again. In this way, if it finishes processing all appending information, it will 
invest.gate whether processing progresses to Step S46 and image information is in appending 
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information. Appending information can know easily by investigating header information [ as 
opposed to each appending information in whether it is an image data ]. 
[0027] When image information is appended, it progresses to Step S47 from Step S46, and it 
investigates whether correlation between image information is performed. This operation gestalt 
is performing with [ between appending information ] relation by the creation date of appending 
information, as Step S34 explained. Therefore, processing in Step S47 is performed by 
investigating whether the creation date of the appending information concerned is registered into 
the header information portion of appending information. 

[0028] If correlation information does not exist, it progresses to Step S43 and the usual display 
and printing are performed. On the other hand, when the associated image information exists, it 
progresses to Step S48. and the image output section 41 displays the image information 
concerned in slide form, or displays and prints it in index form. 

[0029] Drawing 6 is drawing showing the example of a display of the slide form by this operation 
gestalt. At Step S48, the display shown in a display 1 6 by drawing 6 is performed. 
[0030] In drawingjj , 601 is the window of the browser for perusing an E-mail. 602 is a slide 
viewing window and displays the image data appended to the image display field 603 one by one 
in order of the date and time of creation. 604 is a display-mode button and can be set with this 
operation gestalt to the automatic mode automatically performed in time which had renewal of 
the picture in a slide display set up, or the manual mode performed by the manual. 605 and 606 
are the updating directions buttons of the content of a display used at the time of manual mode, 
and if the "front" button 605 is clicked, the image data which has the time of the one past 
rather than an image data present on display will be displayed on the image display field 603. 
Moreover, a click of the "degree" button 606 displays the image data which has the time to 
which one time progressed rather than the image data present on display on the image display 
field 603. 

[0031] Moreover, drawing^ is drawing showing the example of an index display by this operation 
gestalt. Although drawing 7 shows the example of a display to a display 1 6, it may be outputted 
by the printer 17. In drawin g.! . 701 is an index viewing window. 702 is a scroll bar, and when an 
index display is not settled on one screen, it is used for scrolling. The thumbnail generated from 
the appended image data is displayed on the image display field 703. 

[0032] As mentioned above, a slide, and a display and printing in index form are attained, without 
processing especially the still picture image information created by continuous shooting etc. by 
offering the means which associates appending information. 

[0033] Moreover, although considered as the method associated by the date and time of creation 
of an attached file, you may make it associate this operation gestalt in order of the character 
code of the file name of an attached file. Moreover, after directing the file appended at Step S32, 
a user may be made to associate the directed attached file by the manual. For example, the file 
to append is displayed by list form, and when a user changes the display order of a list for the 
file currently displayed on the list by operation of drag and drop etc., it may be made to perform 
**** attachment of an attached file. 

[0034] Moreover, display / printing form does not approach a slide or an index, but may enable it 
to specify other display and printing methods. 

[0035] As mentioned above, since correlation between image information is made by appending 
the still picture image information which carried out continuous shooting to an E-mail according 
to this operation gestalt, the time and effort which processes image information can be saved 
now, and the practical use effect is large. 

[0036] Moreover, since associated image information is displayed and printed in a slide or index 
form according to this operation gestalt, it becomes possible for amount of information with few 
still pictures to give about the same power of expression as an animation, and the practical use 
effect is large. 

[0037] In addition, although image information was appended as appending information on an E- 
mail and the display order was controlled by the above-mentioned operation gestalt using 
correlation information, the information made into a controlled system is not restricted to an 
image. For example, a voice (sound) information file is appended and you may make it control the 
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reproduction sequence by correlation information. 

[0038] In addition, even if it applies this invention to the system which consists of two or more 
devices (for example, a host computer, an interface device, a reader, a printer, etc.), you may 
apply it to the equipments (for example, a copying machine, facsimile apparatus, etc.) which 
consist of one device. 

[0039] Moreover, the purpose of this invention cannot be overemphasized by being attained by 
supplying the storage (or record medium) which recorded the program code of the software 
which realizes the function of the operation gestalt mentioned above to a system or equipment, 
and reading and performing the program code with which the computer (or CPU and MPU) of the 
system or equipment was stored in the storage. In this case, the function of the operation 
gestalt which the program code itself read from the storage mentioned above will be realized, 
and the storage which memorized the program code will constitute this invention. Moreover, 
being contained when the function of the operation gestalt which performed a part or all of 
processing that the operating system (OS) which is working on a computer is actual, based on 
directions of the program code, and the function of the operation gestalt mentioned above by 
performing the program code which the computer read is not only realized, but was mentioned 
above by the processing is realized cannot be overemphasized. 

[0040] Furthermore, being contained, when the function of the operation gestalt which performed 
a part or all of processing that CPU with which the expansion card and expansion unit are 
equipped is actual, and was mentioned above by the processing is realized based on directions of 
the program code, after the program code read from the storage is written in the memory with 
which the expansion unit connected to the expansion card inserted in the computer or the 
computer is equipped cannot be overemphasized. 
[0041] 

[Effect of the Invention] As explained above, it becomes possible to include the information for 
determining the reproduction sequence of appending information in E-mail information according 
to this invention, and information reproduction in desired reproduction sequence can be easily 
performed now. 



[Translation done.] 
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